Multiple mask and boundary scoring R-CNN with cGAN data augmentation for bladder tumor segmentation in WLC videos.

Automatic diagnosis systems capable of handling multiple pathologies are essential in clinical practice. This study focuses on enhancing precise lesion localization, classification and delineation in transurethral resection of bladder tumor (TURBT) to reduce cancer recurrence. Despite deep learning models success, medical applications face challenges like small and limited datasets and poor image characterization, including the absence lack of color/texture modeling. To address these issues, three solutions are proposed: (1) an improved texture-constrained version of the pix2pixHD cGAN for data augmentation, addressing the tradeoff of generating high-quality images with enough stochasticity using the Fréchet Inception Distance (FID) measure. (2) Introducing the Multiple Mask and Boundary Scoring R-CNN (MM&BS R-CNN), a new mask sub-net scheme where multiple masks are generated from the different levels of the mask sub-net pipeline, improving segmentation accuracy by including a new scoring module to refine object boundaries. (3) A novel accelerated training strategy based on the SGD optimizer with the second momentum. Experimental results show significant mAP improvements: the data generation scheme improves by more than 12 %; MM&BS R-CNN proposed architecture is responsible for an improvement of about 1.25 %, and the training algorithm based on the second-order momentum increases mAP by 2-3 %. The simultaneous use of all three proposals improved the state-of-the-art mAP by 17.44 %.

Detection of bladder cancer with feature fusion, transfer learning and CapsNets.

This paper confronts two approaches to classify bladder lesions shown in white light cystoscopy images when using small datasets: the classical one, where handcrafted-based features feed pattern recognition systems and the modern deep learning-based (DL) approach. In between, there are alternative DL models that had not received wide attention from the scientific community, even though they can be more appropriate for small datasets such as the human brain motivated capsule neural networks (CapsNets). However, CapsNets have not yet matured hence presenting lower performances than the most classic DL models. These models require higher computational resources, more computational skills from the physician and are more prone to overfitting, making them sometimes prohibitive in the routine of clinical practice. This paper shows that carefully handcrafted features used with more robust models can reach similar performances to the conventional DL-based models and deep CapsNets, making them more useful for clinical applications. Concerning feature extraction, it is proposed a new feature fusion approach for Ta and T1 bladder tumor detection by using decision fusion from multiple classifiers in a scheme known as stacking of classifiers. Three Neural Networks perform classification on three different feature sets, namely: Covariance of Color Histogram of Oriented Gradients, proposed in the ambit of this paper; Local Binary Patterns and Wavelet Coefficients taken from lower scales. Data diversity is ensured by a fourth Neural Network, which is used for decision fusion by combining the outputs of the ensemble elements to produce the classifier output. Both Feed Forward Neural Networks and Radial Basis Functions are used in the experiments. Contrarily, DL-based models extract automatically the best features at the cost of requiring huge amounts of training data, which in turn can be alleviated by using the Transfer Learning (TL) strategy. In this paper VGG16 and ResNet-34 pretrained in ImageNet were used for TL, slightly outperforming the proposed ensemble. CapsNets may overcome CNNs given their ability to deal with objects rotational invariance and spatial relationships. Therefore, they can be trained from scratch in applications using small amounts of data, which was beneficial for the current case, improving accuracy from 94.6% to 96.9%.

Multi-pathology detection and lesion localization in WCE videos by using the instance segmentation approach.

The majority of current systems for automatic diagnosis considers the detection of a unique and previously known pathology. Considering specifically the diagnosis of lesions in the small bowel using endoscopic capsule images, very few consider the possible existence of more than one pathology and when they do, they are mainly detection based systems therefore unable to localize the suspected lesions. Such systems do not fully satisfy the medical community, that in fact needs a system that detects any pathology and eventually more than one, when they coexist. In addition, besides the diagnostic capability of these systems, localizing the lesions in the image has been of great interest to the medical community, mainly for training medical personnel purposes. So, nowadays, the inclusion of the lesion location in automatic diagnostic systems is practically mandatory. Multi-pathology detection can be seen as a multi-object detection task and as each frame can contain different instances of the same lesion, instance segmentation seems to be appropriate for the purpose. Consequently, we argue that a multi-pathology system benefits from using the instance segmentation approach, since classification and segmentation modules are both required complementing each other in lesion detection and localization. According to our best knowledge such a system does not yet exist for the detection of WCE pathologies. This paper proposes a multi-pathology system that can be applied to WCE images, which uses the Mask Improved RCNN (MI-RCNN), a new mask subnet scheme which has shown to significantly improve mask predictions of the high performing state-of-the-art Mask-RCNN and PANet systems. A novel training strategy based on the second momentum is also proposed for the first time for training Mask-RCNN and PANet based systems. These approaches were tested using the public database KID, and the included pathologies were bleeding, angioectasias, polyps and inflammatory lesions. Experimental results show significant improvements for the proposed versions, reaching increases of almost 7% over the PANet model when the new proposed training approach was employed.

Automatic detection of small bowel tumors in wireless capsule endoscopy images using ensemble learning.

PURPOSE: Wireless Capsule Endoscopy (WCE) is a minimally invasive diagnosis tool for lesion detection in the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the significant amount of acquired data leads to difficulties in the diagnosis by the physicians; which can be eased with computer assistance. This paper addresses a method for the automatic detection of tumors in WCE by using a two-step based procedure: region of interest selection and classification. METHODS: The first step aims to separate abnormal from normal tissue by using automatic segmentation based on a Gaussian Mixture Model (GMM). A modified version of the Anderson method for convergence acceleration of the expectation-maximization (EM) algorithm is proposed. The proposed features for both segmentation and classification are based on the CIELab color space, as a way of bypassing lightness variations, where the L component is discarded. Tissue variability among subjects, light inhomogeneities and even intensity differences among different devices can be overcome by using simultaneously features from both regions. In the second step, an ensemble system with partition of the training data with a new training scheme is proposed. At this stage, the gating network is trained after the experts have been trained decoupling the joint maximization of both modules. The partition module is also used at the test step, leading the incoming data to the most likely expert allowing incremental adaptation by preserving data diversity. RESULTS: This algorithm outperforms others based on texture features selected from Wavelets and Curvelets transforms, classified by a regular support vector machine (SVM) in more than 5%. CONCLUSIONS: This work shows that simpler features can outperform more elaborate ones if appropriately designed. In the current case, luminance was discarded to cope with saturated tissue, facilitating the color perception. Ensemble systems remain an open research field. In the current case, changes in both topology and training strategy have led to significant performance improvements. A system with this level of performance can be used in current clinical practice.

Automatic Segmentation and Detection of Small Bowel Angioectasias in WCE Images.

Angioectasias are lesions that occur in the blood vessels of the bowel and are the cause of more than 8% of all gastrointestinal bleeding episodes. They are usually classified as bleeding related lesions, however current state-of-the-art bleeding detection algorithms present low sensitivity in the detection of these lesions. This paper proposes a methodology for the automatic detection of angioectasias in wireless capsule endoscopy (WCE) videos. This method relies on the automatic selection of a region of interest, selected by using an image segmentation module based on the Maximum a Posteriori (MAP) approach where a new accelerated version of the Expectation-Maximization (EM) algorithm is also proposed. Spatial context information is modeled in the prior probability density function by using Markov Random Fields with the inclusion of a weighted boundary function. Higher order statistics computed in the CIELab color space with the luminance component removed and intensity normalization of high reflectance regions, showed to be effective features regarding angioectasia detection. The proposed method outperforms some current state of the art algorithms, achieving sensitivity and specificity values of more than 96% in a database containing 800 WCE frames labeled by two gastroenterologists.

Brain extraction in partial volumes T2*@7T by using a quasi-anatomic segmentation with bias field correction.

BACKGROUND: Poor brain extraction in Magnetic Resonance Imaging (MRI) has negative consequences in several types of brain post-extraction such as tissue segmentation and related statistical measures or pattern recognition algorithms. Current state of the art algorithms for brain extraction work on weighted T1 and T2, being not adequate for non-whole brain images such as the case of T2*FLASH@7T partial volumes. NEW METHOD: This paper proposes two new methods that work directly in T2*FLASH@7T partial volumes. The first is an improvement of the semi-automatic threshold-with-morphology approach adapted to incomplete volumes. The second method uses an improved version of a current implementation of the fuzzy c-means algorithm with bias correction for brain segmentation. RESULTS: Under high inhomogeneity conditions the performance of the first method degrades, requiring user intervention which is unacceptable. The second method performed well for all volumes, being entirely automatic. COMPARISON WITH EXISTING METHODS: State of the art algorithms for brain extraction are mainly semi-automatic, requiring a correct initialization by the user and knowledge of the software. These methods can't deal with partial volumes and/or need information from atlas which is not available in T2*FLASH@7T. Also, combined volumes suffer from manipulations such as re-sampling which deteriorates significantly voxel intensity structures making segmentation tasks difficult. The proposed method can overcome all these difficulties, reaching good results for brain extraction using only T2*FLASH@7T volumes. CONCLUSIONS: The development of this work will lead to an improvement of automatic brain lesions segmentation in T2*FLASH@7T volumes, becoming more important when lesions such as cortical Multiple-Sclerosis need to be detected.

Automatic T1 bladder tumor detection by using wavelet analysis in cystoscopy images.

Correct classification of cystoscopy images depends on the interpreter's experience. Bladder cancer is a common lesion that can only be confirmed by biopsying the tissue, therefore, the automatic identification of tumors plays a significant role in early stage diagnosis and its accuracy. To our best knowledge, the use of white light cystoscopy images for bladder tumor diagnosis has not been reported so far. In this paper, a texture analysis based approach is proposed for bladder tumor diagnosis presuming that tumors change in tissue texture. As is well accepted by the scientific community, texture information is more present in the medium to high frequency range which can be selected by using a discrete wavelet transform (DWT). Tumor enhancement can be improved by using automatic segmentation, since a mixing with normal tissue is avoided under ideal conditions. The segmentation module proposed in this paper takes advantage of the wavelet decomposition tree to discard poor texture information in such a way that both steps of the proposed algorithm segmentation and classification share the same focus on texture. Multilayer perceptron and a support vector machine with a stratified ten-fold cross-validation procedure were used for classification purposes by using the hue-saturation-value (HSV), red-green-blue, and CIELab color spaces. Performances of 91% in sensitivity and 92.9% in specificity were obtained regarding HSV color by using both preprocessing and classification steps based on the DWT. The proposed method can achieve good performance on identifying bladder tumor frames. These promising results open the path towards a deeper study regarding the applicability of this algorithm in computer aided diagnosis.

Using cystoscopy to segment bladder tumors with a multivariate approach in different color spaces.

Nowadays the diagnosis of bladder lesions relies upon cystoscopy examination and depends on the interpreter's experience. State of the art of bladder tumor identification are based on 3D reconstruction, using CT images (Virtual Cystoscopy) or images where the structures are exalted with the use of pigmentation, but none uses white light cystoscopy images. An initial attempt to automatically identify tumoral tissue was already developed by the authors and this paper will develop this idea. Traditional cystoscopy images processing has a huge potential to improve early tumor detection and allows a more effective treatment. In this paper is described a multivariate approach to do segmentation of bladder cystoscopy images, that will be used to automatically detect and improve physician diagnose. Each region can be assumed as a normal distribution with specific parameters, leading to the assumption that the distribution of intensities is a Gaussian Mixture Model (GMM). Region of high grade and low grade tumors, usually appears with higher intensity than normal regions. This paper proposes a Maximum a Posteriori (MAP) approach based on pixel intensities read simultaneously in different color channels from RGB, HSV and CIELab color spaces. The Expectation-Maximization (EM) algorithm is used to estimate the best multivariate GMM parameters. Experimental results show that the proposed method does bladder tumor segmentation into two classes in a more efficient way in RGB even in cases where the tumor shape is not well defined. Results also show that the elimination of component L from CIELab color space does not allow definition of the tumor shape.

Segmentation of angiodysplasia lesions in WCE images using a MAP approach with Markov Random Fields.

This paper deals with the segmentation of angiodysplasias in wireless capsule endoscopy images. These lesions are the cause of almost 10% of all gastrointestinal bleeding episodes, and its detection using the available software presents low sensitivity. This work proposes an automatic selection of a ROI using an image segmentation module based on the MAP approach where an accelerated version of the EM algorithm is used to iteratively estimate the model parameters. Spatial context is modeled in the prior probability density function using Markov Random Fields. The color space used was CIELab, specially the a component, which highlighted most these type of lesions. The proposed method is the first regarding this specific type of lesions, but when compared to other state-of-the-art segmentation methods, it almost doubles the results.

An adapted double threshold protocol for spastic muscles.

The onset of muscle contraction has been an important element in the understanding of human motor control system as well as in the development of medical devices. This task is problematic in the study of spasticity using surface Electromyography (sEMG). In fact, spasticity is characterized by involuntary muscle contractions that can be seen as both, a non-stationary background if they are weak or a severe non-stationary EMG signal if they are strong. In other hand, these sEMG signals present a very low signal to noise ratio, beyond the added noise that contaminates this signal. The double threshold protocol presumes non-stationary muscle activation under a stationary environment which does not accommodate non-stationary background conditions. Apart from that the Shewhart protocol which makes part of the Double Threshold Protocol (DTP) presumes an initial segment containing only noise which can't be guaranteed under spastic conditions. These are the main causes of failures of state of the art approaches when applied to sEMG spastic muscles. This paper proposes dealing with these constraints by adapting the first threshold to the noise conditions via Signal to Noise Ratio (SNR) estimation, which depends on the severity of the disease. The main idea is tuning the first threshold to low SNR conditions since it is where the DTP most degrades. This tuning is done in sEMG artificially contaminated at different SNRs where the multiple of standard deviation is heuristically determined based on experimentation. Noise is estimated in low energy segments instead of in an initial segment that can be contaminated by involuntary muscle contractions. The proposed algorithm was tested in sEMG signals from the Biceps Braquialis of 13 healthy individuals and in 9315 signals recorded in 23 subjects with spasticity. Improvements of more than 23% were obtained when compared with the classical DTP in moderate to severe spasticity.

Automatic detection of small bowel tumors in endoscopic capsule images by ROI selection based on discarded lightness information.

This paper addresses the problem of automatic detection of tumoral frames in endoscopic capsule videos by using features directly extracted from the color space. We show that tumor can be appropriately discriminated from normal tissue by using only color information histogram measures from the Lab color space and that light saturated regions are usually classified as tumoral regions when color based discriminative procedures are used. These regions are correctly classified if lightening is discarded becoming the tissue classifier based only on the color differences a and b of the Lab color space. While current state of the art systems for small bowel tumor detection usually rely on the processing of the whole frame regarding features extraction this paper proposes the use of fully automatic segmentation in order to select regions likely to contain tumoral tissue. Classification is performed by using Support Vector Machine (SVM) and Multilayer Perceptron (MLP) by using features from color channels a and b of the Lab color space. The proposed algorithm outperforms in more than 5% a series of other algorithms based on features obtained from the higher frequency components selected from Wavelets and Curvelets transforms while saving important computational resources. In a matter of fact the proposed algorithm is more than 25 times faster than algorithms requiring wavelet/curvelet and co-occurrence computations.

Automatic small bowel tumor diagnosis by using multi-scale wavelet-based analysis in wireless capsule endoscopy images.

BACKGROUND: Wireless capsule endoscopy has been introduced as an innovative, non-invasive diagnostic technique for evaluation of the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the output of this technique is an 8 hours video, whose analysis by the expert physician is very time consuming. Thus, a computer assisted diagnosis tool to help the physicians to evaluate CE exams faster and more accurately is an important technical challenge and an excellent economical opportunity. METHOD: The set of features proposed in this paper to code textural information is based on statistical modeling of second order textural measures extracted from co-occurrence matrices. To cope with both joint and marginal non-Gaussianity of second order textural measures, higher order moments are used. These statistical moments are taken from the two-dimensional color-scale feature space, where two different scales are considered. Second and higher order moments of textural measures are computed from the co-occurrence matrices computed from images synthesized by the inverse wavelet transform of the wavelet transform containing only the selected scales for the three color channels. The dimensionality of the data is reduced by using Principal Component Analysis. RESULTS: The proposed textural features are then used as the input of a classifier based on artificial neural networks. Classification performances of 93.1% specificity and 93.9% sensitivity are achieved on real data. These promising results open the path towards a deeper study regarding the applicability of this algorithm in computer aided diagnosis systems to assist physicians in their clinical practice.

Segmentation of small bowel tumor tissue in capsule endoscopy images by using the MAP algorithm.

State of the art algorithms for diagnosis of the small bowel by using capsule endoscopic images usually rely on the processing of the whole frame, hence no segmentation is usually required. However, some specific applications such as three-dimensional reconstruction of the digestive wall, detection of small substructures such as polyps and ulcers or training of young medical staff require robust segmentation. Current state of the art algorithms for robust segmentation are mainly based on Markov Random Fields (MRF) requiring prohibitive computational resources not compatible with applications that generate a great amount of data as is the case of capsule endoscopy. However context information given by MRF is not the only way to improve robustness. Alternatives could come from a more effective use of the color information. This paper proposes a Maximum A Posteriori (MAP) based approach for lesion segmentation based on pixel intensities read simultaneously in the three color channels. Usually tumor regions are characterized by higher intensity than normal regions, where the intensity can be measured as the vectorial sum of the 3 color channels. The exception occurs when the capsule is positioned perpendicularly and too close to the small bowel wall. In this case a hipper intense tissue region appears at the middle of the image, which in case of being normal tissue, will be segmented as tumor tissue. This paper also proposes a Maximum Likelihood (ML) based approach to deal with this situation. Experimental results show that tumor segmentation becomes more effective in the HSV than in the RGB color space where diagonal covariance matrices have similar effectiveness than full covariance matrices.

Small bowel tumors detection in capsule endoscopy by Gaussian modeling of color curvelet covariance coefficients.

This paper is concerned with the classification of tumoral tissue in the small bowel by using capsule endoscopic images. The followed approach is based on texture classification. Texture descriptors are derived from selected scales of the Discrete Curvelet Transform (DCT). The goal is to take advantage of the high directional sensitivity of the DCT (16 directions) when compared with the Discrete Wavelet Transform (DWT) (3 directions). Second order statistics are then computed in the HSV color space and named Color Curvelet Covariance (3C) coefficients. Finally, these coefficients are modeled by a Gaussian Mixture Model (GMM). Sensitivity of 99% and specificity of 95.19% are obtained in the testing set.

Automatic detection of small bowel tumors in capsule endoscopy based on color curvelet covariance statistical texture descriptors.

Traditional endoscopic methods do not allow the visualization of the entire Gastrointestinal (GI) tract. Wireless Capsule Endoscopy (CE) is a diagnostic procedure that overcomes this limitation of the traditional endoscopic methods. The CE video frames possess rich information about the condition of the stomach and intestine mucosa, encoded as color and texture patterns. It is known for a long time that human perception of texture is based in a multi-scale analysis of patterns, which can be modeled by multi-resolution approaches. Furthermore, modeling the covariance of textural descriptors has been successfully used in classification of colonoscopy videos. Therefore, in the present paper it is proposed a frame classification scheme based on statistical textural descriptors taken from the Discrete Curvelet Transform (DCT) domain, a recent multi-resolution mathematical tool. The DCT is based on an anisotropic notion of scale and high directional sensitivity in multiple directions, being therefore suited to characterization of complex patterns as texture. The covariance of texture descriptors taken at a given detail level, in different angles, is used as classification feature, in a scheme designated as Color Curvelet Covariance. The classification step is performed by a multilayer perceptron neural network. The proposed method has been applied in real data taken from several capsule endoscopic exams and reaches 97.2% of sensitivity and 97.4% specificity. These promising results support the feasibility of the proposed method.

Detection of small bowel tumors in capsule endoscopy frames using texture analysis based on the discrete wavelet transform.

Capsule endoscopy is an important tool to diagnose tumor lesions in the small bowel. The capsule endoscopic images possess vital information expressed by color and texture. This paper presents an approach based in the textural analysis of the different color channels, using the wavelet transform to select the bands with the most significant texture information. A new image is then synthesized from the selected wavelet bands, trough the inverse wavelet transform. The features of each image are based on second-order textural information, and they are used in a classification scheme using a multilayer perceptron neural network. The proposed methodology has been applied in real data taken from capsule endoscopic exams and reached 98.7% sensibility and 96.6% specificity. These results support the feasibility of the proposed algorithm.

Cardiac arrhythmia detection by parameters sharing and MMIE training of Hidden Markov Models.

This paper is concerned to the cardiac arrhythmia classification by using Hidden Markov Models and Maximum Mutual Information Estimation (MMIE) theory. The types of beat being selected are normal (N), premature ventricular contraction (V), and the most common class of supra-ventricular arrhythmia (S), named atrial fibrillation (AF). The approach followed in this paper is based on the supposition that atrial fibrillation and normal beats are morphologically similar except that the former does not exhibit the P wave. In fact there are more differences as the irregularity of the RR interval, but ventricular conduction in AF is normal in morphology. Regarding to the Hidden Markov Models (HMM) modelling this can mean that these two classes can be modelled by HMM's of similar topology and sharing some parameters excepting the part of the HMM structure that models the P wave. This paper shows, under that underlying assumption, how this information can be compacted in only one HMM, increasing the classification accuracy by using MMIE training, and saving computational resources at run-time decoding. The algorithm performance was tested by using the MIT-BIH database. Better performance was obtained comparatively to the case where Maximum Likelihood Estimation training is used alone.